Compliance, Utility, and the Future of Credit Union Technology
Opening “Right Thing Right Now”, author Ryan Holiday talks about Hercules at the crossroads.
“For Hercules the choice was between virtue and vice, the easy way and the hard way, the well-trod path and the road less traveled. The same goes for us.”
The road credit unions travel is often one less traveled. It’s harder. It’s a matter of doing work other financial institutions see as unprofitable. Credit Unions serve the under served, bank the under-banked. As Artificial Intelligence (AI) becomes more commonplace in society and systems, there is a tension we can not ignore. AI is everywhere - OpenAI, Anthropic, Google, Meta, Microsoft, Perplexity, Ollama, HuggingFace. The promise? Automate, analyze, detect, personalize. Do it faster! Do it better! Just use AI, get the task done, move on to the next member - serve more. Here’s the paradox: in financial services, every ounce of innovation carries a pound of compliance. We can not adopt tools blindly. We can not ignore data residency, privacy, audit trails, or member trust.
The question isn’t should we use AI — it’s which AI, under which guardrails, and for what purpose?
Online AI: Reach and Velocity
How does this help Credit Union users?
- Attached the transaction journal or financial activity report, As AI:
- Search the transaction journal for member transactions over $10,000.
- Find transactions from a single member account at multiple branches on the same date.
- Report all transactions from a SEG over a range of days
- Copy/Paste the NCUA Compliance Plan for AI, Attach the Credit Union’s Acceptable Use Policy, ask AI -read the attachment and text below. Draft an update to acceptable use to include Artificial Intelligence Risk.
- Summarize a long list of emails
- Search Teams for messages about a topic
- Connect to the CRM and identify new marketing opportunities
- Data leakage: The information you type into AI, the prompt data, could be stored, logged, or transmitted outside the institution’s control.
- Data flow & Data Classification: Regulators frown when you can not report where data goes and what data is used.
- Opaque model training: Models trained on unknown data raise intellectual property and governance concerns.
- Auditability gaps: How do you prove an AI-assisted decision met compliance standards?
Online AI is like an express lane: fast, but you don’t control the tollbooth.
Offline AI: Control and Assurance
- Answer member service questions from internal policies, documentation proprietary to your credit union.
- Automate compliance audits. Upload the SOC reports and run the risk assessment with AI.
- Securely analyze transaction data without leaving the CU’s environment.
- Using the same examples above, except now the documents attached to AI never leave the Credit Union’s network.
Integrate with imaging/document management platforms (AppEnhancer, XP2, Datasafe)
- Find the last 10 statements for Joe Member.
- What is the expiration date on the last scanned ID for Joe Member?
- Are the 1099’s in Nautilus for Joe Member?
What’s the risk?
- Data sovereignty: No sensitive data leaves the credit union’s network.
- Audit trail: Every inference and data source can be logged.
- Customizable governance: Policies can be enforced directly on the model.
- Reduced vendor risk: No third-party AI vendor holding member data.
Offline AI is like running your own private road system: slower to build, but every checkpoint belongs to you.
Online RAG vs Offline RAG
Online RAG Providers (like Azure OpenAI or OpenAI Assistants/Responses API) pull your data into their vector stores, embedding and indexing content to make it searchable.
- Pros: speed, scalability, no need to manage infrastructure.
- Cons: regulatory concerns, unclear data retention policies, vendor lock-in.
Offline RAG (FAISS, pgvector, or custom pipelines with Ollama) keeps all embeddings, indexes, and documents inside your own firewalls.
- Pros: full control, compliance alignment, custom tuning.
- Cons: requires in-house infrastructure, DevOps maturity, and more hands-on expertise.
Balance
For CUs, it often boils down to this: Do you want to trust a vendor’s controls, or own your own compliance pipeline? The truth is most credit unions will need both. Online AI for utility and velocity — things like summarizing board reports, generating marketing copy, or creating member communications. Offline AI for compliance-sensitive work — things like audit testing, risk monitoring, and member data analysis.
The leadership decision isn’t about chasing the flashiest tool. It’s about asking:
- Does this use case require external AI velocity, or internal AI control?
- Can we log and audit every AI-assisted output?
- How do we ensure our AI strategy aligns with examiner expectations?
A Stoic Lens on AI Leadership
Back to “Right Thing Right Now”, Holiday writes:
“Aristotle describes virtue as a kind of craft, something to pursue just as one pursues the master of any profession or skill. “We become builders by building and we become harpists by playing the harp,” he writes. “Similarly, then, we become just be doing just actions, temperate by doing temperate actions, brave by doing brave actions.” Virtue is something we do. It’s something we chose.”
AI is here. We need to practice using it. AI skills will become second nature. Define your organization's framework for success. Build a compliance procedure to measure the risk for your approach. Train your team how to use AI before the bad habits set it. It’s easier to train good habits than it is to break bad habits.
The path forward for credit unions isn’t to choose between compliance and innovation. It’s to clear a path where both coexist. At United Solutions Company, we believe the future of credit union AI is hybrid: responsibly leveraging the scale of online AI while securing the foundation with offline, air-gapped models built for compliance. The winners won’t be those who adopt AI fastest.
